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An algorithm based on simple heuristics is presented for 
an important class of all-binary integer linear programs 
known as the set covering problem. In spite of its very 
special form, the set covering problem has many practical 
applications. Optimal solutions to problems derived from 
these applications are difficult to obtain using known 
methods. Various solution techniques are investigated based 
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some problems. Computational results are reported for 
several large-scale real-world problems and several 
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I. INTRODUCTION 



A. INTRODUCTION 

Set Covering Problems (SCPs) comprise an important class 
of all-binary (0-1) Integer Linear Programs (ILPs). The SCP 
model is well-known and has many practical applications in 
diverse areas such as vehicle routing, facility location and 
capital budgeting. The set covering problem is a theoreti- 
cally difficult problem in that it is NP-complete [Ref. 1 ]. 
However, there exist several methods for obtaining solutions 
to SCEs for quite large real-world problems. In this study, 
heuristics together with branch and bound enumeration are 
tested as a solution method for solving several large-scale 
SCPs. 

There are several reasons for using heuristics with 
branch and bound instead of using cutting plane methods, 
LP-based branch and bound, or seme other technique. First, 
not all researchers have access to good large-scale L? 
systems on which to base cutting plane or branch and bound 
algorithms. Any competent researcher should be able to 
program a heuristic-based method with a modest amount of 
effort. The second reason for wanting heuristic- based 
methods is that more complicated techniques are subject to 
failure as a result of degeneracy, numerical instability and 
slowness. For instance, the systems based on solving the LP 
relaxation, both cutting plane and branch and bound, fail 
when the IPs are difficult to solve because of their size, 
or because of basis structures which are hard to invert, or 
because the LP gives weak bounds. See [Ref. 2] and [Ref. 3]. 
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B. THE SET COVERING PROBLEM 



The SC? is an integer program of the form: 



n 



(1) 


min y\ c x 
3=1 j 3 






n 




(2) 


S . T. £ a. x > b . 

i] ] i 

j=1 


i = 1 # . . . r m 


(3) 


x . binary 


j = n 



where each a = 0 or 1 

i j 

b >0 and integer 

3 

c > 0. 

j 

A minimal cost set of columns must be selected from the 
coefficient matrix A such that the right-hand side t is 
covered or satisfied. Typically, right-hand-side values are 
all Is. Closely related to the SCP is the set partitioning 
problem ( SFP ) where (2) is replaced by (4). 
n 

(4) Y' a x = b , i = 1,...,m. 

. . ij j i 
1=1 

The SPP is the same as the SCP except that each row i must 

be covered exactly b times instead of at least b times. 

i i 

C. OSES OF SCP AND SPP 

Set covering problems and set partitioning problems have 
been studied widely because of their many practical applica- 
tions and simple binary structure. Bausch [Ref. 2] and 
Ealas and Padberg [Ref. 4] give a large collection of 
references to applications which are given below for 
completeness along with some more recent references. Only 
references 6,7,9,16 and 25 are unsighted. 
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A truck routing problem will be described here for the 
purpose of illustrating both the SCP and the SFP. The SCP 
example is described first. The headquarters of the First 
Corps of the Republic of Korea Army has 3 divisions to 
supply using 7 possible delivery routes. It is assumed that 



the cost cf each route is measured in dollars here, tut 
costs could also be measured in time, miles travelled, 
trucks used, etc. The incidence matrix A which is shown in 
Table 1 consists of Is and Os such that 

(l if route j goes through division i 

a i ' = 

(0 otherwise. 



TABLE 1 

Truck Routing Example 



Point 


M 


R2 




R4 


15 


11 


12 


if 1 


1 


0 


1 


1 


0 


0 


0 


if 2 


1 


0 


0 


1 


0 


0 


0 


#3 


1 


1 


0 


1 


0 


0 


0 


#4 


1 


1 


0 


0 


0 


1 


0 


#5 


0 


0 


1 


0 


1 


1 


0 


if 6 


0 


1 


1 


0 


1 


0 


0 


#7 


0 


0 


1 


1 


1 


0 


0 


if 8 


0 


0 


1 


1 


0 


0 


1 


Costs 


7 


8 


10 


12 


6 


5 


5 



A set of truck routes of minimal cost is to be determined in 
such a way that at least one truck route should go through 
each supply point. 

This problem is an SCE 
where A = { a } 

ij 

T 

£=(11111111) and 
c = (78 10 12655). 

Variable x has the value 1 if truck route j is in the 

j 
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mi n i m u m cost set of routes and it has value 0 otherwise. 
The optimal solution is 

rp 

_x — (1/0/ 1/ 0 / 0 / 0 / 0 ) , 

with objective value 17. The solution of the above problem 
using branch and bound will be demonstrated in Chapter 2. 

Suppose, on the ether hand, that trucks are picking up 
supplies and that the net cost of route j is given by 
c = cost of route j - value of supplies at all points in 

j 

route j. Kith this cost structure, no overcovering of any 
row may be allowed and this problem becomes an SPP. The 
optimal solution for the SPP is 

x = ( 1, 0, 0, 0, 1, 0, 1 ) , 

with objective value 18. Since this SPP is a restriction cf 
the previous SCP, it is to be expected that the optimal 
solution to the SPP will be no better than the optimal solu- 
tion to the SCP. 

D. SOLUTION METHODS FOR THE SCP 
1 • Cutting Pla nes 

One method of solving a general ILP is by attempting 
to define the optimal integer solution of the ILP as an 
extreme point of a convex polyhedron generated by the orig- 
inal linear constraints plus some additional constraints 
called "cuts." The technique is applied to the SCP by first 
solving the linear relaxation: 

MIN cx 

(5) S.T. Ax > b 
x > 0. 

Solve this relaxation. If the solution is integer, then the 
solution must be optimal. Otherwise, derive a valid cut. 
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i.e., a linear constraint which is satisfied by all integer 
solutions to SCP, but which is violated by the current non- 
integer solution. Add this contraint to the problem and 
solve the new restricted problem. Continue solving the 
restricted linear programs and adding cuts until either an 
integer solution is obtained or numerical difficulties force 
a halt to the process. 

2 ♦ Eranch a nd B ou nd 



Branch and bound is an optimization technique that 
uses a tree search enumeration approach to the solution of a 
general ILP: 

HIM ex 

(6) S . I. Ax > b 

x > 0 
x integer. 

Following Garfinkel [Eef. 41], denote the set of feasible 
solutions to (6) by 

S = { x | Ax > b, x > 0, x integer } 

Instead of attempting 'to solve directly over S, the set is 
successively divided into smaller sets which have the prop- 
erty that any optimal solution must ce in at least cne of 
the sets. This is called separation and is often illus- 
trated by an enumeration tree with its root node at the top 
of the tree and with restricted subproblems below the root 
(See Figure 2. 1) . Each node of the enumeration tree 
corresponds to a subproblem of (6) . That is, node k is the 
problem 

(7) MIN cx , x is in S 

where S is a subset of S. 
k 

In a binary ILP, S is S with additional constraints which 

k 
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fix certain variables to 0 or 1. As the enumeration 
proceeds further down the enumeration tree, the subsets 
become progressively smaller until it finally becomes 
possible to solve (7) exactly or at least to determine 
whether cr not it contains a potentially optimal solution. 
Subproblems are discarded or "fathomed" when (7) is solved 
or when it is determined that a subset cannot contain a 
solution better than the best known solution to (6) . Upper 
and lower bounds on the optimal solution are calculated for 
each subproblem allowing for more efficient fathoming of 
nodes. The success or failure of branch and bound is 
largely dependent on the accuracy of these bounds. 

Eranch and bound algorithms are often primal in the 
sense that they proceed from one feasible solution to 
another until optimality is verified. In fact they may find 
optimal or near optimal solutions at an early stage in the 
enumeration process and spend the majority of the time veri- 
fying optimality by improving bounds. 

E. IMPLEMENTATION 

1 • Introduc ti on 

Most large-scale mathematical programming problems 
have special structure which is exploited in the implementa- 
tion of mathematical programming solvers. Examples of such 
special structure are sparsity of the constraint matrix and 
the frequent occurrence of some coefficient values. To take 
advantage of this structure, the computer programs written 
for this study are written as subroutines embedded in a 
large-scale optimization test bed called the X System or 
simply XS [Eef . 42 ]. XS is designed to solve linear 
programming problems, 0-1 programming problems, nonlinear 
programming problems and mixed 0- 1 /line ar /nonlinear 

programming problems. XS uses sparse matrix techniques 
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common to many mathematical programming systems. A mere 
specialized system using binary vectors to represent the A 
matrix [Eef. 43] might be faster for some problems but less 
flexible. 

2 • In 2 irt Da ta Format 

In this study, to make data manipulation easy and 
convenient, the data format described by Bausch [Eef. 2] is 
used since this format has many advantages for large-scale 
problems. The advantages are as follows. 

a. It is compact. 

b. Storage requirements are easily calculated. 

c. Data generation problems are simplified. 

d. Column manipulation of data input is made easy 
since all information for each column is contiguous. 

e. This column format is easily generated by commer- 
cially available (MPS) problem generation systems. 

The data input format consists of three sets of card images: 

a. Problem dimensions. Format (316) (One Card) 

M = Number of rows 

N = Number of columns 

NZEL = Number of non-zero elements. 

b. Constraint ranges. Format (2A4, 2216.8) (H Cards) 

IE = Eow index i 

EL = Lower range limit b 

i 

EU = Upper range limit (always CO) . 

c. Column Data. (N or More Cards) (2A4, F14.3, 1015) 

JC = Column index j 

C = Column cost coefficient c 

3 

NCE = Number of non-zero elements in the column 
IE row addresses of non-zero coefficients. 

If NCE is greater than 9, additional column cards 
are needed to hold the row addresses for that 
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cards is 



column. The format for additional column 
(20X, 1015). 

An example of this data format is shown in Appendi 
the truck routing example of Table 1. 

3. Test Problems 



Eight test problems a 
These problems consist of 
(American, Bus, Tiger and True 
iems (Steinerl, Steiner2, Stei 
and Steiner2 are problems devi 
anteed to require extensive e 
branch and bound since the LP 
and Steinr2a are Steinerl and 
tively . 



re evaluated in 
four real-worl 
k) , and four arti 
nr 1 a and Steinr2a) 
sed in [Ref. 493 a 
numeration when us 
bounds are so weak 
Steiner2 transpos 



th is 
d 

fici 

• 

nd a 
ing 

ed , 



Some of the problems are, in fact, pur 
However, we have converted these problems in 
reasoning that the derived SCP should still be rep 
tive of a true SCP. The characteristics of these 
are shown in Table 2 where NZEL is the total nu 
nonzeros in the constraint matrix and NCE is the 
number cf nonzeros in each column. All these 



except Truck are 


typical set 


covering problems wh 


right-hand sides 


eg ual to 1 . 


The Truck 


proble 


general right-hand 


side. All 


of these test 


probl 


run on an IBM3033 


under V M/CHS. 


Computation 


times 


ir. the following 


chapters are 


accurate to 


the nu 


decimal places sho 


wn . 







x A for 



thesis, 
probl ems 
al prob- 
Steiner 1 
re guar- 
IP- based 
Steinr 1 a 
respec- 

e SPPs. 
to SCPs 
resen ta- 
pr oblems 
mber of 
av erage 
problems 
ich have 
m has a 
ems were 
reported 
mber of 
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TABLE 2 

Problem Dimensions 



Problem 


Rows 


Columns 


NZEL 


NCE 


Model 


American 


95 


9318 


57293 


6.0 


SPP 


Bus 


56 


530 


3339 


6.3 


SPP 


Steiner 1 


1 17 


27 


352 


13.0 


SCP 


Steiner 1 A 


27 


117 


351 


13.0 


SCP 


Steiner2 


330 


45 


991 


22.0 


SCP 


Steiner2A 


45 


330 


991 


22.0 


sc? 


Tiger 


160 


636 


4134 


6.7 


SPP 


Truck 


239 


4752 


30075 


8.0 


SCP 
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II. BRANCH AND BOUND ENUMERATION 



A. INTRODUCTION 

In this chapter we introduce "branch and bound" enumera- 
tion which will be used in chapter 5 to solve SCPs. Branch 
ar.d bound is an optimization technique that uses tree 
enumeration together with upper bounds and lower bounds on 
the objective function. These bounds help to accelerate the 
fathoming process and reduce enumeration. In this chapter, 
we describe branch and bound in terms of a minimizing binary 
IIP, and discuss the importance of good bounds and good 
branching strategies. In a binary ILP, a separation is 
effected by fixing a binary variable to its possible values, 
0 and 1. Thus, every separation of a problem is, in fact, a 
partition of the problem into two subproblems. 

The discussion of branch and bound is limited to a 
"depth-first" search or exploration of the enumeration tree 
since this is the method that was used in this research. 
More general techniques are possible ( See Garfinkel and 
Nemhauser [Ref. 43]. ) but these all require substantially 
more storage and general overhead. Most commercial branch 
and bound packages utilize a depth-first search. 
Depth-first search simply means that when a separation is 
defined, one of the nodes created by the separation is imme- 
diately selected to be the next subproblem, and when a node 
is fathomed, the enumeration always backtracks to the most 
recently created live node which is the "twin" of a node 
already explored. 

There are two important parts of any branch and bound 
algorithm. First, good upper and lower bounds must be 
obtained on the optimal solution. The closer the bounds are 
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to the optimal solution, the fewer nodes (subproblems) must 
be enumerated. There exist several ways of obtaining upper 
and lower bounds on an SC?. Methods for obtaining computa- 
tionally simple lower bounds will be described in Chapter 4. 
Upper bounds on the optimal solution are given by heuristi- 
caliy obtained feasible solutions to the ILP. Heuristic 
solution methods for the SCP will be described in detail in 
the next chapter. 

The second important part of the branch and bound algo- 
rithm is the method of determining which variable to fix 
first when a separation is defined at a node in the tree and 
whether that variable should be fixed to 0 or 1. This 
selection process is called a "branching strategy." 
Branching strategies are dependent on the methods being used 
for obtaining upper and lower bounds and the actual strat- 
egies to be used will be discussed in chapter 5. It seems 
obvious that if a good guess can be made as to which vari- 
ables must be in the optimal solution, then fixing one of 
those variables first to T would be a good branching 
strategy. Of course, guessing is very difficult; otherwise 
we would have guessed the solution to the whole problem. 
Another likely strategy may be to select the least attrac- 
tive variable in the incumbent and set that variable to 0. 
Unfortunately, as will be seen in chapter 5, no single rule 
seems to work well on all problems and a certain amount of' 
case-ly-case experimentation is necessary. 

B. FATHOMING C1ITERICN 

To accelerate the enumeration process and save computing 
time we need a criterion to decide whether or net a 
subproblem should be discarded at a certain point of the 
algorithm. Suppose that several steps of the enumeration 
have already been performed and that a subproblem at a 
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particular node in the tree is being considered. Let BEST 
denote the smallest feasible objective value found thus far 
in the enumeration. Clearly, BEST is an upper bound on the 
optimal solution to the IIP. The feasible solution 
corresponding to BEST is called the "incumbent.” 

Now, let CLBND denote a lower bound on the optimal solu- 
tion to the IIP given the restrictions at the current node. 
CLBND is defined to be infinity if no feasible solution to 
the IIP can be found given the current restrictions. let 
CUBND denote a upper bound on the optimal objective value 
corresponding to a feasible solution to the IIP given the 
current restrictions. If BEST > CUBND, let BEST = CUBND and 
let the corresponding solution be the new incumbent. Now, 
the efficiency of branch and bound enumeration is based on 
the fact that explicit enumeration need not be extended 
below the current node if the "fathoming criterion" is met: 

Fathoming criterion: CLBND > 3EST. 

For problems with integer costs, fractional values for CLBND 
should be rounded up to the nearest integer. 

From a computational viewpoint, it is useful to split 
the above test into two tests, however. First, compute 
CLBND and test if CLBND > BEST; if it is, the node is 
fathomed. If not, only then compute CU3ND, update BEST if 
appropriate and repeat the test. This avoids some 
unnecessary computation of upper bounds. 

C. SEPABATION AND BRANCHING 

"Branching" describes the process whereby an unexplored 
subprcblem is selected for exploration, i.e., upper and 
lower bounds are computed for the node and the node is 
either fathomed or separated. "Separation" is the process 
whereby the current subproblem is separated into two or more 
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subsubproblems, at least one of which must contain the 
optimal solution to the current subproblem if such a solu- 
tion exists. 

In a binary LP using depth-first search, branching and 
separation are intertwined. A separation is always a 
partition based on fixing a specific variable to 0 or to 1. 
After a separation one of the live nodes just created must 
be immediately selected for branching. If a node is 
fathomed, the most recently created live node must be 
selected for exploration. 



D. AIGOBITHH AND EX APPLE 
1 * Algorithm 

The following branch and bound algorithm uses depth- 
first exploration of the enumeration tree. The logic is 
exactly that used in the programs written for this thesis. 

Algorithm: Depth-First Branch and Bound 
STEP 0. (Initialization) 

Let BEST =OQ>, STACK = <j) . 

STEP 1. Compute CUBND given restrictions defined by 
STACK. 

If CLBND > BEST, go to step 5. 

STEP 2. Compute CUBND given restrictions defined by 
STACK. 

If CUBND < BEST, then let BEST = CUBND and save 
incumbent. 

If CLBND > BEST, go to step 5. 

STEP 3. (Branching) Select an unfixed variable j to fix 
and determine whether to fix it to 0 or 1. 

STEP 4. Put vertex j in STACK with information indica- 
ting whether it is fixed to 0 or 1 and that 
its twin has not yet been explored. 

Go to step 1. 



22 



STEP 5. (Backtrack) 

If STACK = <j) , then go to step 7. 

STEP 6. Remove j from top of STACK. 

If its twin has been explored, go to step 5. 
Otherwise, put j back on STACK fixing j to the 
complement of its previous value and noting that 
its twin has already been explored. 

Go to step 1. 

STEP 7. Termination 

If 3EST =C0 , there exists no feasible solution. 
Otherwise, current incumbent is optimal. 

End of Algorithm: Branch and Bound 

2 . Exa mple 



The example below illustrates the above algorithm on 
the SCP defined in Table 1. Lower bounds on the solutions 
at each node are obtained by finding a feasible solution to 
the dual of the LP relaxation of the SC? (See section E in 
Chapter 4.). The upper bounds are obtained by using an 
’’addition heuristic" which successively adds columns to a 
partial cover until a complete cover is obtained (See 
section C in Chapter 3.). Separation is effected by 
randomly selecting a variable x among all variables not 

j 

in the current solution obtained by the addition heuristic. 

The branch corresponding to x = 1 is explored first. 

3 

The enumeration tree is shown in Figure 2. 1. 



a . 
b . 



Initialize: 
Node 0: CIBND 
CLBND 
CDBND 

CUBND 



EEST = 00 
= 17.0 

< BEST so continue. 

= 18.0, X = ( 1,0, 0,0, 1,0,1 ) 

“0 

< BEST so let BEST = C03ND and X 

-0 



becomes the incumbent. 
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is selected 



c. Node 1 



d. Node 2 



e. Node 3 



f. Node 4 



CIBND < BEST so variable x 

2 

for branching. Fix variable x^ to 1 first. 

Given that x = 1, CLBND = 25.0. 

2 

Since CLBND > BEST, backtrack to the twin 

of this node which has not been explored. 

Given that x = 0, CLBND = 17.0. 

2 

CLBND < BEST so continue. 

CUBND = 18.0. No improvement over incumbent. 

Since CLBND < BEST, select x for branching. 

5 

Set x = 1 . 

5 



Given that x = 0 and x = 1, CLBND = 18.0. 

2 5 

Since CLBND > BEST, backtrack to the twin 

of this node which has not been explored. 

Given that x = 0 and x =0, CLBND = 17.0. 

2 5 

Since CLBND < BEST, continue. 

CUBND = 17.0 for = ( 1 , 0 , 1 , 0 , 0 , 0 , 0 ). 

Since CUBND < EEST, let BEST = CUBND and 

let X be the new incumbent. 

-4 

Since CLBND ^ BEST, backtrack. 

No live nodes exist, so the current incumbent 

X is optimal with objective value 17.0. 

-4 
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(17.0, 18.0) 




Note: Pairs are (CLEND, BEST) 



(17.0, 18.0) 



x = 0 
5 



(17.0, 17.0) 



Figure 2.1 Enumeration Tree for Truck Routing Example. 
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III. heuristic solution techniques 



A. introduction 

Two basic heuristic techniques exist for obtaining good 
feasible solutions to SCPs: "addition" heuristics and 

"deletion" heuristics. These two heuristics are used in 
this study for the purpose of generating solution sets and 
upper bounds. These heuristics are not guaranteed to solve 
the SCP optimally but can be used to get good upper bounds 
on the optimal solutions which are essential in the branch 
and bound enumeration. Feasible solutions to the SCP are 
easily obtained because of the SCP’s greater than or equal 
to constraints and nonnegative constraint matrix. 
Computational results are given in section D. 

B. ADDITION HEURISTIC 

An addition heuristic begins with the infeasible solu- 
tion x=0 and successively sets to 1 that variable x which 

3 

myopically minimizes effective cost. The effective cost 

associated with x is c /p , where p is a penalty which in 

j j j 3 

some way reflects the amounts of in feas ibility currently 

being contributed by x =0. The addition heuristic can be 

3 

stopped when a feasible solution is obtained but it is 
possible that the cover produced is not minimal and a second 
phase should be added which deletes any columns in the cover 
which are redundant. 
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algorithm: 
Input : 
Output 
STEP 0 



STEP 1 



STEP 2 



Addition Heuristic 

The SCP matrix and vectors A, c and b. 
Upper bound to SCP solution. 
"Initialization” 



1 {1,2,...,m} , I — (1,2,. ..,n}, 

J ’ = t' = b, CU3ND = 0. 

For each column j£J 

Compute a penalty p > 0. 

3 

Let h = number of nonzeros in column j . 

j 

If b’ < 0 or J is empty, go to step 3. 
Otherwise, let 



c 

3 

i = arqmin — 
0 j|j p 

j 




J ’ = J ’ + j 

0 



CUBND = CU3ND + c . . 

For each i such that a = 

i;1 o 

Let b ’ = b’-l. 
i i 



1 



If b' = 0 (update column sums) 
i 

Fcr each j such that a = a = 

ij i3 o 

Let h = h -1. 

3 3 

If h =0, let J = J - j . 

j 

If b' < 0, go to STEP 3. 

Otherwise , 

For each column j€J 

Update penalties p if necessary. 

3 



Go to step 1. 



1 , 
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STEP 3. "Generating minimal cover" 

For each each j € J 1 

If column j is redundant 

Let CUB ND = CUBND - c . 

j 

let J 1 = J' - j. 

STEP 4. "Termination" 

Halt. ‘ If J 1 is a cover, CUBND is an upper hound 
on the SCP. 

Otherwise, no feasible solution exists. 

End of Addition Heuristic 

Two different penalty functions have been tested with 

the above addition heuristic: p = h and p = k where k 

j 3 3 3 3 

is the initial column sum (number of nonzeros in the column' 

which is never updated. Kovac [Ref. 44] suggests using k 

3 

as part of a heuristic for obtaining both upper and lower 

bounds on the optimal solution to an SCP. The results 

obtained using this penalty are not reported here, however, 

since they are so poor. It should be noted that more 

complicated penalty functions could certainly be defined 

such as p = log (h ) . In addition, instead of selecting the 
3 3 

minimum c /p the minimum of a more general functional form 
3 3 

g (c , p ) could be selected. 

3 3 

C. DELETION HEURISTIC 

A deletion heuristic begins with the feasible solution 

x=1 and successively sets to 0 that variable x which 

3 

mvopically minimizes effective profit c /p . Here c is some 

3 3 3 
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penalty reflecting the amount of overcovering which column j 
is contributing. The deletion heuristic stops when no 
alditional variables can be set to 0 without creating an 
inf easibiility implying that the cover obtained is minimal. 
The following algorithm carries out the above ideas. 

Algorithm: Deletion Heuristic 

Input: The SCP matrix and vectors A, c and b. 

Output: Upper bound to SCP. 

STEP 0. "Initiali 2 ation" 

I = {1/2/. • • , m} , J = {1/2/. ../n}/ 

n 

jt = (j), t' = b, CUBND = £ c. 



3=1 



For each i£l. 



let h = number of nonzeros in row i. 
i 

For each j£J, 

let h = number of nonzeros in column j. 
3 

compute a penalty p > 0. 

3 

STEP 1. If b' < 0 or J is empty, go to STEP 4. 

c 

3 

g = argmax — 

0 j£J p 

j 

STEP 2. For each i such that a =1, 

lj o 

If b’ = h , go to STEP 4. 
i i 



STEP 3. J = J - j 



0 



CUBND = CUBND - c 



For each row j such that a = 1 

ij 

let h = h - 1. 
i i 

Go to STEP 1. 



2 S 



For each a =1 let J' = J' * j 

i;i o 

For each a =1 let b' = b' - 1 
ij i i 

If b = 0, update column sum h letting 

i j 

J = J - j for any h =0. 

3 

If J)' <0, go to STEP 6. 

For each column j€J. 

Update penalties p if necessary. 

j 

Go to STFE 1. 

"Termination" 

Halt. If J' is cover, CU3ND is an upper bound 
on the SCP. 

Otherwise, no feasible solution exists. 

End of Algorithm Deletion Heuristic 

The deletion heuristic has only been tested using p =h . 

3 3 

The comments on the possible use of more general functional 
forms in the addition heuristic are also valid here, but 
have not teen tested. 

D. COMPUTATIONAL RESULTS 

An addition heuristic and a deletion heuristic have been 
coded for purposes cf comparison. The results are summa- 
rized in Table 3. As can be seen, the addition heuristic is 
faster than deletion heuristic, but the upper bound from the 
addition heuristic is not as good as that obtained by the 
deletion heuristic except for problems Steiner 1 A and Truck. 
Although these results tend to indicate that the deletion 
heuristic is better, the true test of usefulness in solving 
SCPs will have to wait until chapter 5 where the heuristics 
are embedded in a branch and bound algorithm. 



STEP 4. 



STEP 5. 



STEP 6. 



30 



TABLE 3 

Computational Results of Upper Bounds 



Problem 


Addition 


Deletion 




Value 


Time 


Value 


Ti me 


A merican 


3.532 


0.35 


3.364 


35. 13 


B us 


5.253 


0.03 


5. 192 


0.64 


S teiner 1 


19.0 


0.00 


19.0 


0.00 


Steinr la 


9.0 


0.00 


10.0 


0.00 


S teiner2 


32.0 


0.00 


30.0 


0.01 


Steinr 2 a 


16.0 


0. 00 


16.0 


0.24 


Tiger 


59.264 


0. 11 


59. 173 


0. 92 


Truck 


367.64 


1 . 19 


389.62 


53. 35 



The above heuristic techniques are one-pass methods and 
are the cr.ly methods implemented in this research. 
Multiple-pass methods exist and are mentioned here for 
completeness. The most straightforward multiple-pass method 
is called the "1-opt" method [Ref. 51]. This method first 
uses an addition or deletion heuristic to obtain a minimal 
cover. Then, each column in the current solution is checked 
against the columns not in the solution to determine if a 
one-for-one exchange can be made which maintains a feasible 
cover while reducing total cost. The 1-opt method is an 
example cf an "exchange heuristic. " The basic idea can be 
extended to a k-way exchange resulting in the "k-opt" 
method. 
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IV. ICWER BOUNDS ON THE SCP 



A. ISTRCDUCTION 

Several methods cf finding lower bounds for solutions to 
the SCP are described in this chapter. Getting good lower 
bounds on the optimal solution to an SCP is critical if 
optimal solutions are to be obtained using branch and bound. 
Lower bounds are also necessary if the accuracy of nonop- 
timal solutions is tc be bounded when branch and bound fails 
to find the optimal solution. There are many possible 
methods of obtaining lower bounds on the SCP, all based on 
solving some relaxation of the the associated ILP. Several 
methods have been coded and are compared to decide which 
method should be employed within the branch and bound 
enumeration. Although these bounds have not been used for 
solving any SPPs, it should be noted that they are all valid 
for the SPP since the SCP is a relaxation of the SPP. 

A feasible solution to the dual of the LP relaxation of 
the SCP provides one easily obtainable lower bound. Also, a 
column partitioning method is given in which the SCP is 
partitioned into small SCPs which can be solved exactly and 
whose solution values may be summed to give a lower bound. 
Another lower bound which is tested is the sum of all of the 
minimal row covering fractions. All these methods are coded 
and computational results compared in section E. 

B. DUAL LP RELAXATION LOWER BOUND 

One obvious relaxation of the SCP which can be solved to 
obtain a lower bound is the linear programming (LP) 
relaxation. This technique has been used successfully in 
many cases [Ref. 45]. But a problem with the LP lower bound 
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is that it may be difficult to solve the LPs associated with 
many SCPs. (See for example Bausch [Ref. 2] and Salkin and 
Koncal [Ref. 3]. ) This is true because the LP may be quite 
large, highly degenerate and have a basis structure which is 
numerically unstable. However, it is possible to get a 
quick lower bound on the SCP by just finding a feasible 
solution to the dual of the LP relaxation of the SCP since. 



w (L? ) < w* (LP ) = v*(LP ) < v* (SC?) 

D D P 



where w (LP ) = objective value for a feasible solution 
D 

tc the dual of the LP relaxation, 

w* (LP ) = optimal value of the dual LP relaxation, 
D 

v* (LP ) = optimal value of primal LP relaxation, 
P 



and 

v* (SCP) = optimal value of the SCP. 

Letting 1_ denote a row vector of ones, the dual tc the LP 
relaxation of the SCP is 



max b u - 1v 



s.t. A u - Iv < C 
u > 0, v > 0. 

The dual LP looks very similar to the SCP itself if the dual 
variables _v are set to zero, and a simple method for 
obtaining a feasible solution can be devised in a way that 
is similar to the greedy addition heuristic for the SCP. 



Algorithm: Dual LP Relaxation Lower Bound (DLPRLB) 

Input: SCP coefficient matrix and vectors A, c and h. 
Cutput: Lower bound CLBND to the SCP 
STEP 0. "Initialization" 

CL BHD = 0 

I = {1,2,... ,m} , J = {1/2,. ..,n) 

For each i£I, let h = number of nonzeros 

i 



33 



m row 1 



STEP 1. If I is empty, go to STEP 5. Else, let 

b 

i 

i = argmax 
0 i€l h 



I = I - i 



'0 



STEP 2. c 



= nin c 

3 n j : a . . = 1 3 

0 

j = j - j 0 

STEP 3. CLBND = CIBND + c b 

3 o 1 

STEP 4. Eor each row i such that a =1, 

i3 o 

For each column j such that a = 1, 

ij 



Update cost coefficients 
0 



c = c - c 

j j j 



Fepeat STEP 1. 

STEP 5. "Termination" 

Halt. CIEND is a lower bound on the SCP. 
End of Algorithm DLPEIB 



At each iteration of the algorithm,. dual variable u 

i 

would be set to c . . The actual values of the dual 

j o 

variables need not be retained, however, since the value of 
the dual objective function is just b T u. 

Hey [Ref. 45] gives a somewhat more complicated method 
for finding a feasible solution to the dual relaxation of 
the SCP and this is tested along with the method described 
above . 
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PARTITIONING LOW EE BOUND 



C. 



Marsten [Ref. 50] gives a method for finding a lower 
bound on the SCP by solving subproblems of the SCP defined 
on certain partitions of the coefficient matrix. It is 
easier tc describe the method, however, in terms of a maxim- 
ization problem. 

Erown, McBride and Wood [Ref. 46] give a way to calcu- 
late an upper bound using' a partition of the columns for a 
problem of the form: 



MAX cx 
S.T. Ax < b 

x binary. 

They use the bound for estimating the size of the maximum 
embedded generalized network in an LP constraint matrix 
where c = _1, b = 2, and A is the transpose of the 0-1 inci- 
dence matrix associated with an LP constraint matrix. Their 
bound is found as follows. 

Let A and A be a partition of the columns of A and let z, 
1 2 

z and z be the sclution of the maximization problem on 
1 2 



A, 



A 

1 



and A respectively. 



Then, z < z + z . 

1 2 



If A ^ is intelligently chosen, z^ can be computed exactly. 

Then, A becomes A and the partitioning scheme is recur- 

2 

sively repeated until z^ can be easily solved also. 

The SCP can be converted to a maximization problem by 

substituting variables 1-y for x and multiplying the objec- 
tive function by -1. Thus, it is not hard to see that a 

lower bound on the SCP can be obtained using the above 

method. Cf course, the method can be applied directly 
without making the substitution. The partition of the 
columns 

A is created with respect to an arbitrary row i. The 
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containing 



columns of A are those columns of row i 

1 

nonzeros. The minimum cost among those cost coefficients 
contributes an additive term to the lower bound. The vari- 
ables included in the partition are never considered again 
and all rows with nonzero intersections in the partition are 
also never considered again. 



Algorithm: Partitioning Lower Bound 

Input: SCP coefficient matrix and vectors A, c and b. 
Output: Lower bound CLBND to the SCP 
STEP 0. "Initialization" 

For each i€I, 

let h = number of 'nonzeros in row i 
i 

STEP 1. If I is empty, go to step 4. 

Let i = argmin h 
0 i£I i 



I = I - i 



STEP 2. (Find the b minimum c in row i .) 

i j 0 

0 

Let j = argmin c . 

0 j:a y =1 j 

V 

Let CLBND = CLBND + c 

Let c = oo 

Let b = t - 1. 
i i 
0 0 

If b_ > 0, go to step 2. 



STEP 3. For each j' such that a = 1, 

V 



For each i such that a = 1 

i V 
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let b = b - 1 
i i 

If b = 0, let I = I - i. 
i 

Go to step 1. 

STEP 4. "Termination" 

Halt. CIEND is a lower bound to the SCP. 
End of Algorithm Partitioning lower Bound 



D. KOVAC’S LOWER BOUND 



Consider the basic model of the SCP with all right-hand 
side values equal to 1. 
n 



(8) 



MIN Yj ' 
3=1 



x 

j 



n 

S. T V a x > 1 i = 1 , . . . , m 

3=1 lj 3 

x binary j = 1 , . . . , n. 

j 

The following lemma [Ref. 44] provides a lower bound fo 
the above SCP. 



Lemma Denote the optimum of problem (8) by Z 
then 
* m 

(9) Z > £ f . = F 

i = 1 

where f is the minimal covering fraction of row i 
i 



0 

f = MIN { r | 0 < j < n, a =1}. 

i 3 ij 

(Proof) Define the following new problem. 
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n m 

(io) min E E r . y . k 

j=1 k=1 3 3 

n m 

s.t E E g. . v y .. * 1 

: , t1 iDk 3k 

3=1 k=1 

3 — 1 1/* • • , n , k — 1/ 



i = V 



y binary 
jk 



• • • / in 



where 



9 ijk 



a if j = k 

i 3 



0 



otherwise. 



Tc any feasible solution x of the SCP, there corresponds 
a solution of problem (10) in such a way that the objective 
function values are equal. Specif ically, for each x_. = 1, 

v =1 for all k. On the other hand the minimum of problem 

jk 

(10) is obviously F. This proves the statement (9) of the 



lemma . 

It is easy to extend the Kovac bound to problems with 
general right-ha nd- sides by using 
m 



F 



E 

i= 1 



b.f . . 
i i 



E. COMPUTATIONAL RESULTS 

Computational results for the lower bounds described 
above are reported here for all the test problems (See Table 
4. "DNR" indicates did not run.). It is not clear that the 
tightest lower bound for the complete problem will perform 
the best in the branch and bound enumeration, but some posi- 
tive correlation is to be expected. The computational speed 
of the bound is also a consideration in branch and bound and 
some increase in speed may be traded for a loss in accuracy. 
Thus, the results given here are a guide to which bound will 
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he most effective in the branch and bound algorithm but only 
testing with that algorithm can determine true effective- 
ness. The dual LP relaxation method (DLPRL3) appears to be 
the best in the problems bus, Steinerl, Tiger and American. 
Actually, both DLPRLB and Hey's method do outstandingly well 
on Bus. Kovac's heuristic appears to be superior to the 
ether bounds in Truck, Steinrla, Steiner2 and 3teinr2a. Tor 
Steinrla and Steinr2a, the bounds are tight indicating that 
if good heuristic solutions can be obtained, the branch and 
bound algorithm should terminate very quickly. 
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V. COMPUTATIONAL EXPERIENCE AND DIFFICULTIES 



A. RESULTS 

All of the computational results reported in this 
section are for SCPs. The algorithms performed very well on 
some of the problems but not on the others. At first, small 
test problems were used to check, the correctness of the 
algorithms. All algorithms worked well on these small prob- 
lems. The algorithms were then tested on the eight problems 
described in chapter 1. To solve these problems, we have 
used five methods to see which method is more effective than 
the others. The descriptions of the methods follow. 

Method 1: Lower bound: Dual L? relaxation. 

Upper bound: Addition heuristic. 

Separation: j = argmin c /h among all 
0 3 3 

x = 1 in current solution. 

3 

Branching: x = 1 first. 

3 0 

Method 2: Lower bound: Dual LP relaxation. 

Upper bound: Addition heuristic. 

Separation: j = argmax c /h among all 
0 3 3 

j not in current solution. 

Branching: x =0 first. 

3 

Method 3: Lower bound: Dual LP relaxation. 

Upper bound: Deletion heuristic. 

Separation: argmin c /h among all variables 

j 3 

j not in current solution. 

Branching: x = 1 first. 

3 



Method 4: Lower bound: Dual LF relaxation. 

Upper bound: Deletion heuristic. 

Separation: argmax c /h among all variables 

j j 

j not in current solution. 
Branching: x =0 first. 

j 

Method 5: Lower bound: Kovac's 

Upper bound: Addition heuristic. 

Separation: argmin c /h among all variables 

j j 

j not in current solution. 

Branching: x = 1 first. 

3 

As illustrated in Table 5, three problems were not 
solved optimally. We denote the actual percentage with 
respect to the optimal value as a "^OPT" and the provally 
optimal percentage as "%POPT. " %P0PT denotes the amount by 
which were able to prcve that the best solution found varied 
from the optimal solution without knowing the optimal solu- 
tion. This value is obtained by changing the CLBND > EEST 
tests in the branch and bound algorithm to 
CLBND > BEST-EPS where EPS is. an allowable amount of error. 
If the branch and bound algorithm then halts, it fellows 
that the incumbent solution is within 1 00 % • (BEST + EPS) /B EST 
of the optimal solution. 
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All these problems except Truck are typical set covering 
problems which have right-ha ni-side values equal to 1. The 
Truck problem has the general right-hand-side form shown in 
Equation (2) . The computational results are summarized in 
Table 5. These results are the best of the various solution 
methods tried. "Previous Results" indicate either those 
times reported by Bausch or the times we recorded using the 
methods of Bausch. Running times on Steiner2 are for 10000 
nodes only; optimality was not proven in either our or 
Bausch's computation. Table 6 shows the comparison between 
the different solution techniques on each of the problems. 
The problems marked with * were not solved optimally within 
1 minute of CPU time. 
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B. EXAMPLE 



One cf the results of the tests was that the deletion 
heuristic usually produces better feasible solutions than 
the addition heuristic both initially and further down the 
tree. This leads to the enumeration of fewer nodes with the 
deletion heuristic. Unfortunately, it does not lead to 
faster times because the deletion heuristic is so much 
slower than the addition heuristic. For example, using 
method 4 which includes the deletion heuristic, it is 
possible to solve Bus after exploring only 15 nodes. Using 
method 1 with the addition heuristic requires developing 53 
nodes to solve Bus. Cn the other hand, the method using the 
deletion heuristic requires 4.24 seconds to solve the 
problem while the method using the addition heuristic 
requires only 0.82 seconds to solve the problem. 

In order to illustrate the actual behavior of the algo- 
rithm, the enumeration for Bus is shown below for two 
different methods, method 2 and method 4. For these two 
methods, the enumeration trees are sufficiently small to be 
shown. The entire trees generated for Bus are displayed ir. 
Figure 5.1 and Figure 5.2. Note that for both methods, the 
optimal solution is found at the second node of the 
enumeration tree. Most of the running time of the algorithm 
is spent proving optimality after the optimal solution is 
found . 



Figure 5.1 Method 2 on Bus 



s(S) 

^ UJ — 

<?T; 
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Figure 5.2 Method 4 on Bus. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



The trar.ch and bound enumeration method using heuristi- 
cally obtained upper and lower bounds works well cn seme 
problems and poorly cn others. Solution times are better 
than the times using the methods described by Bausch on 
certain problems but other problems could not be solved to 
optimality in a reasonable amount of time. The algorithm is 
largely dependent upon the quality of bounds obtained, and 
in certain instances these bounds are not very good. 

The greedy addition heuristic used here does not perform 
as well as might be heped and the deletion heuristic, which 
performs better, is too slow to use in most cases. Other 
addition heuristics should probably be tested which select 
that column j minimizing some function 

g (c ,h ), where g (c , h ) is some function other than c /h 
3 3 3 3 3 3 



such as c /log (h ) . 

j j 



In fact, Vasko and Williams [Ref. 51] 



have had some success selecting randomly from a number of 
such functions, albeit on randomly generated problems. They 
also utilize a 1-opt heuristic. Future research should 
examine the use of this and other exchange heuristics, 
particularly in conjunction with the addition heuristic 
since it may be possible to significantly improve upon the 
solutions obtained without sacrificing much computational 
speed . 

The lower bound from the dual LP works quite well on 
some problems and poorly on others, notably Truck. Of 
course, the LP- based bound did not work well on Steinerl or 
Steiner2 since those problems were concocted so as to have 
very poor LF relaxations. The high speed of computation for 
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this lower hound does allow rapid investigation of a large 
number of nodes, however. In the Steiner problems, we 
expected that branch and bound enumeration using Xovac’s 
lower bound might work better than the other lower bounds 
since the value of the initial lower bounds were stronger 
than the other bounds as shown in Table 4. Unfortunately, 
the guality of the bound does not improve rapidly enough as 
the enumeration proceeds. Additional research is needed to 
generate better heuristic solution sets and lower bounds. 
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APPENDIX A 

DATA FORMAT FOR TRUCK ROUTING EXAMPLE 



7 23 

0. 1 0000000D+0 1 
0. 100000 00D+ C 1 
0. 10000000 D+01 
3. 1 0000000D+01 
0. 10000000D+01 
0. 1000 000 0D +01 
0. 1 00000 00D+ 01 
0. 10000000D+ Cl 

7.000 

8.000 

10.000 

12.000 

6.000 

5.000 

5.000 



0. 10000000D+21 
0. 10000000D+21 
0. 10000000D+21 
0. 10000000D+21 
0. 10000000D+21 
0. 10000000D+21 
0. 10000000D+21 
0. 10000000D+21 

4 12 3 

3 3 4 6 

5 15 6 

5 12 3 




4 

7 8 

7 8 
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